System and method for stringed instruments&#39; pickup

ABSTRACT

A method for stringed instruments&#39; pickup, the method comprising a step of: capturing mechanical vibrations of at least one string; and converting them to a signal representative of a string&#39;s current state, the method being characterized in that the capturing comprises the following steps: capturing, using an image capturing device, image frames comprising views of at least one musical instrument&#39;s string in a still state; storing the captured image as a reference of a still state; capturing ( 500 ), using the image capturing device, image frames comprising views of at least one musical instrument&#39;s string in a vibrating state; storing the captured image as a reference of a vibrating state; comparing ( 520 ) the still state reference with a vibrating state reference in order to find ( 530 ) amplitude of vibrations of each string as well as frequency of each string vibrations based on amplitude height in pixels with reference to the still state and determining a frequency of each vibrating string on the basis of the number of pixels between two nuts of at least half-period of a given periodic function.

TECHNICAL FIELD

The present invention relates to a system and method for stringedinstruments' pickup. In particular, the present invention relates to apickup device, which is a transducer that captures mechanical vibrationsfrom stringed instruments such as an electric guitar, an electric bassguitar, a harp or an electric violin, and converts them to an electricalsignal that is representative of a string current state.

BACKGROUND OF THE INVENTION

Known solutions of converting vibrations of strings, in musicalinstruments, into electric signals involve typically a use of coils.

A ferromagnetic string passing through the magnetic field induceselectric current in the coil wound on a pole of a permanent magnet. Theelectric current induced in this way has the same frequency as thefrequency of the vibrating string. Obtained in this way signal isamplified and played back by a speaker at much higher power.

This solution is very prone to electromagnetic interference due to thehigh sensitivity of such solutions needed for high efficiency inconverting musical instrument string vibrations into electric signals.

Moreover, the need of connecting such sound sensors to potentiometersfor volume and tone control as well as to the outlet socket poseadditional problems for interference-free sound signal transmissionespecially at the stage where the level of signals is very small andprone to unwanted signal interference.

One additional limitation is a necessity of having a ferromagneticstring in a musical instrument, which is needed to induce electriccurrent in the pick-up coils. This way of transforming the vibration ofstrings into electric signal is not suitable for instruments havingnylon or gut strings which do not induce electric current in coil basedpick-up systems.

Another way of transforming vibrations of physical elements intoelectric signal involves the use of microphones or piezo-electricdevices. Both solutions transform vibrations of air or mechanicalelements onto electric signals, which are subject of further processingor amplification.

With piezo-electric sound pick-ups or microphones there is no necessityof having ferromagnetic strings, but instrument producers face problemswith unwanted feedback, crosstalk or noise interference at the earlystage of sound signal processing and amplification.

Other, less popular and practically hardly implemented, known solutionsof converting the vibrations of strings into electric signals apply theuse of optic sensors as described in U.S. Pat. No. 8,546,677, where anemitted light stream or laser beam or any other form of electromagneticwaves of different length is interrupted by the vibrating string and theoptical sensor receiving the interrupted in this way light/infraredstream or reflections are the source of the string frequency response.

In a way, this solution is similar to the coil based pick-up with thedifference that instead of the magnetic field interrupted by thevibrating ferromagnetic string it is the emitted light that getsinterrupted by the vibrating string and received by a suitably placedlight sensor.

Although the employment of light or any other form of electromagneticwaves of various length decreases the unwanted noise interferencesignals being a major problem with magnetic coil, microphone, or piezobased solutions, it cannot produce any form of information about the wayof sound generation related to the physical application of forceinducing the string vibration.

This information, called in music “sound articulation”, can only beheard when the sensed and amplified sound is reproduced by a speaker. Inelectronic music sound modules, information about sound articulation isdescribed by the duration and dynamics of sound. This may be regarded asa limitation in musical expression especially when musicians useelectronic stringed instruments and MIDI based sound modules.

An advantage of keyboard instruments over stringed instruments in MIDIelectronic sound systems have led to the search of other means ofcollecting information about the produced by a stringed instrumentsound.

In view of the above, the aim of the development of the presentinvention is an improved or at least alternative system and method forstringed instruments' pickup.

SUMMARY AND OBJECTS OF THE PRESENT INVENTION

An object of the present invention is a method for stringed instruments'pickup the method comprising a step of: capturing images of mechanicalvibrations of at least one string; and converting them to a signalrepresentative of a string's current state, the method beingcharacterized in that the capturing comprises the following steps:capturing, using an image capturing device, image frames comprisingviews of at least one musical instrument's string in a still state;storing the captured image as a reference of a still state; capturing,using the image capturing device, image frames comprising views of atleast one musical instrument's string in a vibrating state; storing thecaptured image as a reference of a vibrating state; comparing the stillstate reference with a vibrating state reference in order to findamplitude of vibrations of each string as well as frequency of eachstring vibrations based on amplitude height in pixels with reference tothe still state and determining a frequency of each vibrating string onthe basis of the number of pixels between two nuts of at leasthalf-period of a given periodic function.

Preferably, the rate with which the frames are delivered is controlledby a clock and is at least twice as high as the highest frequency agiven musical stringed instrument is able to produce.

Preferably, before the comparing step, an image processing step isexecuted where the irrelevant elements of the captured scene as well asthe elements which carry meaningful information are identified.

Preferably, the method further comprises a step wherein the obtainedinformation of amplitude and frequency of at least one vibrating stringis matched with corresponding MIDI messages that are capable of drivingexternal MIDI sound modules or sound synthesis modules.

Preferably, the given periodic function is a sine or cosine.

Preferably, the image capturing device viewing axis creates an anglewith the still string axis in the range of 0 to 90 degrees.

Preferably, determining the frequency includes calculating a time perpixel on the basis of a known calibrating frequency of a vibratingstring, and a number of pixels between two nodes (613) of half-periodand applying the following formula:

$f_{Sound} = \frac{1}{2 \cdot ( {t_{{Node}\mspace{14mu} 2} - t_{{Node}\mspace{14mu} 1}} )}$

Preferably, the comparing step is based on a correlation of a stillstring axis, and the camera viewing axis, and the most distant pixel'strajectory axis.

Preferably, the information about the sound frequency and amplitude,produced by the stringed instrument, takes place in time intervals equalto the full vibration period of the highest tone a musical stringedinstrument is able to produce.

Another object of the present invention is a computer program comprisingprogram code means for performing ail the steps of thecomputer-implemented method according to the present invention when saidprogram is run on a computer.

Another object of the present invention is a computer readable mediumstoring computer-executable instructions performing all the steps of thecomputer-implemented method according to the present invention whenexecuted on a computer.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other objects of the invention presented herein, areaccomplished by providing a system and method for stringed instruments'pickup. Further details and features of the present invention, itsnature and various advantages will become more apparent from thefollowing detailed description of the preferred embodiments shown in adrawing, in which:

FIG. 1 presents a diagram of the system according to the presentinvention;

FIG. 2 presents examples of camera orientation;

FIG. 3 presents examples of shapes of strings observed by a high speedvideo camera;

FIG. 4 shows details of video data analysis module;

FIG. 5A depicts a general overview of the method according to thepresent invention;

FIG. 5B depicts a more detailed diagram of the method presented in FIG.5A focusing only on blocks 500, 510, and 520;

FIG. 5C depicts a more detailed diagram of the method presented in FIG.5A focusing only on blocks 530, 540, and 550;

FIG. 6 presents views of groups of pixels representing a stringvibrating at its third harmonic;

FIG. 7 presents views of groups of pixels representing a stringvibrating its fundamental frequency and;

FIG. 8 shows relevant equations.

NOTATION AND NOMENCLATURE

Some portions of the detailed description which follows are presented interms of data processing procedures, steps or other symbolicrepresentations of operations on data bits that can be performed oncomputer memory. Therefore, a computer executes such logical steps thusrequiring physical manipulations of physical quantities.

Usually these quantities take the form of electrical or magnetic signalscapable of being stored, transferred, combined, compared, and otherwisemanipulated in a computer system. For reasons of common usage, thesesignals are referred to as bits, packets, messages, values, elements,symbols, characters, terms, numbers, or the like.

Additionally, all of these and similar terms are to be associated withthe appropriate physical quantities and are merely convenient labelsapplied to these quantities. Terms such as “processing” or “creating” or“transferring” or “executing” or “determining” or “detecting” or“obtaining” or “selecting” or “calculating” or “generating” or the like,refer to the action and processes of a computer system that manipulatesand transforms data represented as physical (electronic) quantitieswithin the computer's registers and memories into other data similarlyrepresented as physical quantities within the memories or registers orother such information storage.

A computer-readable (storage) medium, such as referred to herein,typically may be non-transitory and/or comprise a non-transitory device.In this context, a non-transitory storage medium may include a devicethat may be tangible, meaning that the device has a concrete physicalform, although the device may change its physical state. Thus, forexample, non-transitory refers to a device remaining tangible despite achange in state.

As utilized herein, the term “example” means serving as a non-limitingexample, instance, or illustration. As utilized herein, the terms “forexample” and “e.g.” introduce a list of one or more non-limitingexamples, instances, or illustrations.

DESCRIPTION OF EMBODIMENTS

The present invention relates to an image capturing device for vibrationfrequency recognition in musical Instruments or non-musical devices(Visual Pickup).

The present solution is based on an employment of a video camera andpicture analysis for a determination of string pitch (sound frequency)and the way of sound articulation (the way of making the string vibrate)and can provide a bit wider range of stringed instrument output data,which can be used in the later stage of sound synthesis and processingin MIDI based music related systems.

One additional advantage apart from the elimination of the undesirednoise in the early stage of electric signal processing is eliminatingthe need of having a string with ferromagnetic qualities that arenecessary to make the traditional coil based pickups detect thevibration of strings. In this way instruments having nylon or gutstrings, typical for a harp, will also be suitable for the applicationof the presented here idea.

According to the invention presented in FIG. 1, a string frequencydetector and converter (110) is based on a very high speed video cameramodule (111), a video data analysis module (112) for analyzing aposition of each string, in pre-determined time video frames, andestablishing elementary parameters of the 25 produced sound so that aparticular response may be generated by an output module (113) such asthe frequency and amplitude of the vibrating string in a musicalinstrument (in this particular case-harp). Such output signal may beprovided to a MIDI sound module (120).

The very high speed video camera module (111) may be a Fantom Miro 320Sor FASTCAM Mini AX200 or the like. Their size and speed properties makethem suitable for their application in sound producing instruments,where picture analysis is the source of information about the frequencyand the qualities of the produced sound.

In another embodiment of the invention, each string may have anassociated, separate camera. For example a guitar pickup having sixstrings and six cameras.

In yet another embodiment of the invention, each string may have onlyoptic module associated with particular string which is furtherconnected via optic fiber with the image capturing device. The installedon the stringed musical instrument optic module may be of single ormultifocal type.

A very high speed video camera (111) is mounted on the stringedinstrument to obtain VIEW 1 of a string or a set of strings, as shown inFIG. 2.

FIG. 2 presents a camera (111) oriented with respect to a string (200)in three different positions. While the string (200) maintains itsorientation axis (201), the camera orientation axis (202A, 202B, 202C)may vary in a given setup.

The way of mounting the camera (111) should ensure that the cameraviewing axis be as close to the string axes as possible. In this way thecamera has the most convenient position to observe the vibrations ofstrings. An example of the alignment of a camera and a still string maybe that the camera viewing axis and the string axis are positioned suchthat an angle equal or lower than 90 degrees is formed. Similarly, in apreferred embodiment a distance of the camera from the string(s) is in arange of few millimeters to ten centimeters.

FIG. 3 presents examples of shapes of strings (200) observed by a highspeed video camera (111) depending on the way of making the stringvibrate.

In the case of a guitar, item (303) is a neck string nut while item(304) is a bridge string nut. Item (301) is a guitar tuning peg whileitem (302) is a bridge pin securing a string in a bridge.

In case of a harp, item (303) is equivalent to the harp's bridge pin orstationary string nut pin, item (304) is equivalent to the harp's eyeletand (302) is equivalent to the harper's knot securing the harp's stringin the sound box and item (301) is equivalent to harp's tuning pin.

Picture A shows a still string (200), picture B shows the deflections ofa string when vibrating at its whole length (305A, 305B), called inmusic at its fundamental frequency, picture C shows the deflections of astring vibrating at its 2nd harmonic (306A, 306B), picture D showsdeflections of a string vibrating at its 3rd harmonic (307A, 307B) whilepicture E shows deflections of a string vibrating at its 4th 10 harmonic(308A, 308B).

A camera capable of taking a series of pictures at the speed of at leasttwo times higher than the frequency of the highest tone produced by astringed instrument, according to the Nyquist—Shannon sampling theoremis able to provide reliable information about the frequency of thevibrating string when taking into account the known frequency of takingpictures (Nyquist rate) of the vibrating string or other source ofsound.

The analysis of the deflection degree on the taken series of picturescan provide further information regarding the loudness of the sound andthe way it fades out in time. What could be interesting to derive fromthe series of images captured by the High Speed Video Camera, is the waya string vibration is initiated. A camera picture can also providemeaningful information which after suitable picture processing canindicate the articulation of the produced by a string sound andinfluence the qualities of the sound generated by the MIDI sound modulestriggered by a stringed instrument such as a harp.

Since the present size of Very High Speed Cameras is still relativelybig and not suitable for their application for example in a classicalguitar, a violin, or a mandolin, instead of having the camerasthemselves mounted on the instrument, it is also possible to installonly the optic part of the Very High Speed Video Cameras (VHSVC) on astringed instrument and connect the optic part via optic fiber to theVHSVC located at a distance from the stringed instrument together withother MIDI sound modules or amplification systems.

FIG. 4 presents a diagram of the system according to the presentinvention, in particular the video data analysis module (112).

The system may be realized using dedicated components or custom madeFPGA or ASIC circuits. The system comprises a data bus (401)communicatively coupled to a RAM memory (431) and a non-volatile FLASHmemory (432). Additionally, other components of the system arecommunicatively coupled to the system bus (401) so that they may bemanaged by a controller (410).

The memory (432) may store computer program or programs executed by thecontroller (410) in order to execute steps of the method according tothe present invention. Additionally, the memory (432) may store anyconfiguration data of the system. Such configuration data may includeinformation regarding one or more of the following:

-   -   sound frequencies associated with sounds produced by each string        in a given stringed instrument;    -   images of strings in still states as reference for any computing        purposes in an Image Interpreter Unit;    -   MIDI messages as specified by the MIDI standard;    -   nominal names of strings associated with corresponding sound        frequencies strings produce when they are tuned. Each string in        a musical instrument is featured by its name which corresponds        to the sound the string produces when it obtains its nominal        tension (in other words when the given string is tuned;    -   parameters describing possible lengths of strings (identified        groups of pixels as string lengths);    -   lengths of strings and corresponding frequencies when strings        are tuned to their nominal values. In a harp, each tuned via        tuning pins string may be additionally shortened by a set of        tuning discs driven by the harp's pedals. By pressing pedals the        instrumentalist shortens the length of the harp's strings        (shortens the length between the first and the last nut) by a        pre-defined length and obtains respectively higher sound. The        action of the tuning discs could be compared to pressing a        string on a fret in a fretted stringed musical instrument. In        fretted stringed instruments it is possible to pre-define        frequencies produced by a tuned string for each fret. E.g. if a        tuned string e1 in a classical guitar is pressed on the first        fret it will produce f1 sound. When the same tuned string is        pressed on the second fret, it will produce f#1 sound, on the        third fret, it will produce g1 sound. Similarly, for each string        in a fretted stringed musical instrument configuration data        stored in system memory may hold pre-defined frequencies        corresponding to each string pressed on each fret. This        pre-definition of sounds may allow to avoid the frequency        recognition process in the cases of fretted stringed        instruments.    -   device viewing axis (202A, 202B, 202C), string axes (201), and        the axis (608) describing the trajectory of the identified pixel        (616A), (616B), (616C), (616D),    -   angular correlation of axes (201), (202A), (202B), (202C) and        (608);    -   resolution of images delivered by image capturing devices;

image capturing device viewing perspective (202A), (202B), (202C)compensation parameters for image pixels;

-   -   samples of predefined acoustic effects available for matching        with user defined groups of pixels derived from image        decomposition and analysis.

A clock (450) is responsible for generating timing control of takingpictures by the camera (111). Each taken picture shall be associatedwith a time stamp. A suitable command triggering the camera (111) may beissued via a wired (404) or wireless (405) communication interface by atime controller (414).

Data received from the camera (111) may be processed by a digital signalprocessor (420) in order to obtain a frame sample to be stored in memoryfor further reference by the controller (410).

The controller (410) comprises an image processing manager (411)responsible for controlling an image interpreter unit (412) and an imagerecognition unit (413).

The Image Recognition Unit (413) is responsible for identifyingmeaningful elements of the captured scene that during a further stagecan be a source of information of a string frequency, string vibrationinitiation, or other here undefined features. The elements may include,for example, recognition of a collection of pixels depicting particularstrings of the musical instrument. Another recognized element of thecaptured scene may include string name identification as each string ina musical instrument is featured by a name corresponding to a particularsound the string achieves when it obtains its designed nominal tension.

Yet another feature recognized by the Image Recognition Unit may be theidentification of those elements of the captured scene that areirrelevant to producing sound parameters. Eliminating the irrelevantelements of the scene helps to limit the amount of data subject totransfer and consequently to shorten the time needed to recreate thefrequency of the vibrating string without the sense of delay that mayappear if the time from the physical sound initiation moment till themoment the sound is reproduced exceeds 30 ms.

The Image Recognition Unit may also define various groups of pixels ofthe captured scene which change in time at the speed indicating playersactivity rather than musical instrument's frequency response.

The identified captured scene elements in the Image Recognition Unit(413) are delivered to the Image Interpreter Unit (412) where theidentified scene elements are further translated onto various soundparameters.

This module is responsible for calculating the frequency of thevibrating string and outputting the results of the calculation atintervals shorter than 30 ms. Also, this module translates theidentified scene elements into other sound features typical for a soundsuch as articulation or the way of string vibration initiation. ImageInterpreter Unit may either match the identified scene elements withpre-defined sound features or produce the sound features each time itreceives meaningful information from the Image Recognition Unit.

FIG. 5A presents a diagram of the method according to the presentinvention. The method starts at step (500) where a very high speed imagecapturing device captures image frames containing views of musicalinstrument strings. The rate with which the frames are delivered iscontrolled by the Clock (450) and is at least twice as high as thehighest frequency a given musical stringed instrument is able toproduce. Image frames containing the views of strings are thendecomposed in image processing step (510) where the irrelevant elementsof the captured scene 15 as well as the elements which carry meaningfulinformation are identified. This stage of the process removes theirrelevant elements of the captured scene.

Subsequently, step (520) allows to differentiate various groups of thecaptured scene, tag them and make them the subject of further analysis.Image Interpreter (530) processes the meaningful elements of decomposedimage frames. Processing the chosen meaningful elements of thedecomposed image frames leads to establishing the parameters of soundsproduced by the musical instrument. Establishing sound parameters takesplace in the step (540) where the obtained information (of amplitude andfrequency) is matched with corresponding MIDI messages that are capableof driving external MIDI sound modules. Sound parameters obtained in(540) may also be presented in such a way which will make them suitablefor influencing new sound synthesis.

In particular, there is executed comparing of the still state referenceimage with a vibrating state reference image in order to find amplitudeof vibrations of each string as well as frequency of each stringvibrations based on amplitude height in pixels with reference to thestill state and determining a frequency of each vibrating string on thebasis of the number of pixels between two nuts of a half-period of agiven periodic function.

A more detailed diagram of the method presented in FIG. 5A is presentedin FIG. 5B and FIG. 5C. Delivered by (500) or (501) image frames aresubject of image processing (510) which could be further illustrated ina more detailed way by three stages (511), (512), and (513).

Delivered images containing views of strings are decomposed into groupsof pixels representing strings, relevant scene background, irrelevantscene background, and groups of pixels known in MPEG compressionstandards as macro blocks featured by their motion vector indicating theinstrumentalist's playing action.

These macro blocks are further tagged for example as instrumentalist'sfingers, finger tips, nails, plectrum, palm. Any user defined namescould be assigned to the selected groups of pixels (macro blocks).

The system also identifies those groups of pixels in the captured scenethat have no influence on the process of sound parametersidentification. Those groups of pixels in the captured scene are removedto limit the bitrate ratio and consequently shorten the time lapsingfrom the moment of physical sound initiation till the moment of soundreproduction. The identification of irrelevant groups of pixels takesplace in the step (512).

Next, having the decomposed image frames where various relevant groupsof pixels have been identified and tagged, groups of pixels representingseparate strings are obtained (513). This stage of the method alsopresents groups of pixels identified and tagged as instrumentalist'svarious playing means which affect the way a particular string vibrationis initiated. Image object recognition generally presented by (520) isfurther described by (521), (522), (523) where (521) carries out theanalysis of changes of the groups of pixels representing strings inconsecutive image frames, (522) identifies which group of pixels (whichmacro blocks) in consecutive image frames represent particular way ofstring vibration initiation, and (523) decides which user defined groupsof pixels in consecutive image frames can be assigned user defined soundparameters.

Image interpreter (530) obtains the results of analysis performed by by(521), (522), (523) and correlates the results allowing the (540) toform sound parameters having information whether the particular sound isgenerated by plucking with the use of a plectrum, finger tip, nail,hammer on or pull off or other user defined 5 technique and combiningthis with the sound frequency or suitable MIDI message carryinginformation about sound parameters or with other messages capable ofdriving any sound synthesis module parameters. In this particularembodiment the sound parameters comprise the sound fundamentalfrequency, its corresponding MIDI sound number together with possiblePitch Bend messages (541A), sound amplitude or MIDI sound velocityparameters (541B), or the number of harmonics (541C) a string isvibrating at.

The results of analysis released by (522) and (523) may further beprocessed by Image interpreter (530) to release commands and messagesinfluencing MIDI sound module or sound synthesis module settings asindicated by (552A), (552B) or (553).

One possible method of obtaining the information about frequency, of avibrating string in a stringed instrument, may include the followingsteps.

Image of a tuned still string is captured and kept in memory as areference. Prior to writing in memory, the captured image is analyzedand decomposed. A group of pixels representing a given string, in astill state, is identified and kept in memory.

Next, image with strings is calibrated in such a way that a string of aknown vibration frequency vibrating at its non fundamental frequency iscaptured in a frame (see FIG. 6, 610A). Next, the system identifiescommon pixels of two groups of pixels. One group of a still string (607)and the second group of the vibrating string (610A). Common pixelsdenote nodes (613) of the vibrating string.

Knowing the calibrating frequency of the vibrating string, and a numberof pixels between two nodes (613) of the half-period, applying formulafrom FIG. 8 (801), a time per pixel (606) is calculated. Obtaining thevalue (606) ends the calibration process. The higher resolution of theimage and consequently the number of pixels available in the image themore precise is the calibration process and further other frequenciesidentification performed on the basis of the calibration and calculatedtime per pixel.

In order to calculate other frequencies of strings, vibrating at theirnon-fundamental frequencies (803A), there may be applied the sameformula known from FIG. 8 (801), where having common pixels denotingnodes of the vibrating strings, knowing the number of pixels between thenodes, and having the calculated Constant time per pixel (606), theremay be obtained a frequency of the string (803A) vibrating at itsnon-fundamental frequency.

The constant value (606) allows to calculate the string vibrationfrequency when the string is vibrating at its fundamental frequency.Formula (802) allows to compute the sound frequency (803B). In thismethod of string vibration frequency identification, the (606) Constantis applied to calculate the time the group of pixels representing thevibrating string moves from their one extreme deflection from its stillstate (710A) to the opposite maximum deflection from the still state(710D).

Another method of string vibration frequency identification, that doesnot require the calibration process, may comprise the following steps

An image of a still string is captured and kept in memory as reference.Prior to writing in memory the captured image is analyzed anddecomposed. A group of pixels representing a given string in a stillstate (707) or (607) is identified and kept in memory.

The image capturing device delivers images of a string with the ratehigher or equal to 6 [kHz]. The exposure time of the captured imagesallows to deliver unmoved (sharp and focused) images of deflectedstrings like in (710A), (710B), (710C), (710D), (610A), (610B), (610C),(610D).

The method identifies a pixel in the group of pixels representing thedeflected string, which is located further away (616A) from the axisdrawn by the group of pixels representing the still string and assignsto that pixel an electric value proportional to the number of pixels onthe axis drawn perpendicularly (608) to the still string axis from theposition of the most distant pixel till the still string axis asindicated by (612), (611B), (612C) or (611D).

The axis (608) which is perpendicular to the still string (201), (607),(707) axes may additionally be checked at least every second pair ofcaptured images to verify if axis (608) does not pivot on the stillstring axis (201), (607), (608). If pivoting action is detected, thecorresponding correction co-efficient is applied in assigning anelectric value for the pixel being the subject of analysis. Thecorrection co-efficient is the result of angular co-relation of (608)with (201) or (607) or (608), and with (202A) or (202B) or 202C).Theangle of both axes (608) and (607) or (707) may additionally becorrelated with the image capturing device viewing axis (202A), (202B),or 202C). For the above description it has been assumed that thetrajectory of the most deflected pixel located on the group of pixelsrepresenting the vibrating string moves along (608) axis, which isperpendicular to the still string axis (607) or (707) where the imagecapturing device viewing axis (202C) creates a right angle (203C) withthe still string axis (201), or (607) or (707).

A similar analysis takes place on each consecutive image, delivered bythe image capturing device, until another pixel is identified whoseposition is further away from the still string axis than the position ofthe pixel being the subject of current analysis.

In this way, a series of discrete values is obtained, each timeproportional to the number of pixels between the most distant positionof a pixel from the still string axis and the point of juncture of thetwo axes (608) and (607) or (707), or (201).

Additionally, the values (612), (611B), (612C), or (611D) obtain + or −sign depending on the side the pixels are located with reference to thestill string axis (607) or (707) or (201).

In this way, a series of values are created which represent samples ofsound frequencies or in other words discrete-time signals derived fromthe vibrating string depicted in FIG. 6 or FIG. 7.

The discrete-time signals are stored in memory until the system releasesthe 5 information about the frequency of the vibrating medium here(610A), (610B), (610C), (610D) or (710A), (710B), (710C), (710D). Theinformation about the frequency of the produced sound is released by adigital to analogue converter being the part of the output module (113).

Forming and releasing the information about the sound frequency andamplitude 10 produced by the stringed instrument in the digital toanalogue converter takes place in time intervals equal to the fullvibration period of tones a musical stringed instrument produces. Thetime interval should not exceed 30 milliseconds to avoid the sense ofdelay which may appear if the sound formation takes place later than 30milliseconds from the time of physical sound initiation. The informationabout the frequency produced by the stringed instrument sound mayfurther be matched either with corresponding MIDI messages which couldbe used for driving an external MIDI sound modules (120).

Yet another way of deriving the information about the string vibrationfrequency is based on comparison. The comparison of the string lengthagainst the held-in-memory information on the length of the string, itsnominal name and its frequency at its nominal tension. This methodrequires the calibration process where the image capturing deviceprovides the image of strings in their still state at their nominaltension.

The image is analyzed, decomposed and the image of each string as agroup of pixels is held in memory. Each group of pixels representingparticular string is assigned a corresponding name and frequencyassociated with the string name at its nominal tension according to thefollowing table:

The table below lists music sounds and their corresponding frequenciesdivided into octaves which may be kept in memory as reference.

Octave Sound Frequency [Hz] Sub C2 16,351598 Contra Cis2 17,323915 D218,354048 Dis2 19,445437 E2 20,601723 F2 21,826765 Fis2 23,124652 G224,499715 Gis2 25,956544 A2 27,500000 Ais2 29,135235 B2 30,867707 ContraC1 32,703196 Cis1 34,647829 D1 36,708096 Dis1 38,890873 E1 41,203445 F143,653529 Fis1 46,249303 G1 48,999430 Gis1 51,913088 A1 55,000001 Ais158,270471 B1 61,735413 Great C 65,406392 Cis 69,295658 D 73,416193 Dis77,781747 E 82,406890 F 87,307059 Fis 92,498607 G 97,998860 Gis103,826175 A 110,000001 Ais 116,540942 B 123,470827 Small c 130,812784cis 138,591317 d 146,832385 dis 155,563493 e 164,813780 f 174,614118 fis184,997213 g 195,997720 gis 207,652351 a 220,000002 ais 233,081883 b246,941653 First Line c1 261,625568 cis1 277,182634 d1 293,664771 dis1311,126987 e1 329,627560 f1 349,228235 fis1 369,994427 g1 391,995440gis1 415,304702 a1 440,000005 ais1 466,163766 b1 493,883306 Second Linec2 523,251136 cis2 554,365268 d2 587,329542 dis2 622,253974 e2659,255121 f2 698,456470 fis2 739,988853 g2 783,990880 gis2 830,609404a2 880,000009 ais2 932,327533 b2 987,766613 Third Line c3 1046,502272cis3 1108,730535 d3 1174,659084 dis3 1244,507948 e3 1318,510241 f31396,912940 fis3 1479,977706 g3 1567,981760 gis3 1661,218807 a31760,000018 ais3 1864,655065 b3 1975,533225 Fourth Line c4 2093,004544cis4 2217,461071 d4 2349,318168 dis4 2489,015895 e4 2637,020483 f42793,825880 fis4 2959,955412 g4 3135,963520 gis4 3322,437615 a43520,000036 ais4 3729,310131 b4 3951,066451 Fifth Line c5 4186,009088cis5 4434,922141 d5 4698,636335 dis5 4978,031791 e5 5274,040965 f55587,651761 fis5 5919,910824 g5 6271,927040 gis5 6644,875230 a57040,000073 ais5 7458,620261 b5 7902,132902 Sixth Line c6 8372,018176cis6 8869,844283 d6 9397,272670 dis6 9956,063582 e6 10548,081930 f611175,303521 fis6 11839,821649 g6 12543,854081 gis6 13289,750460 a614080,000145 ais6 14917,240522 b6 15804,265803

The image capturing device delivers images of strings at the rate of 6kHz or higher. When a string vibration is initiated, the image capturingdevice begins to deliver a series of images where the group of pixelsrepresenting strings takes a deflected position. The higher theamplitude of the vibrating string, the bigger the difference is obtainedwhen two images, one of a string in a still state and the other of astring in deflected position, are compared.

By combining two parameters, the length of the string and the result ofimage comparison where the image of a still string is compared with theimage of a deflected string, the information on the sound frequency andits duration may be derived.

Using the aforementioned methods of image analysis one can obtaininformation capable of driving MIDI sound modules or devices specializedin sound synthesis not only from stringed instruments featured bystrings of ferromagnetic qualities, but also from stringed musicalinstruments that are featured by nylon or gut strings. Using imageanalysis as the source of information about the observed sound,additionally allows to avoid presently known methods prone to externalnoise and interference. Therefore, the invention provides a useful,concrete and tangible result.

The presented invention captures image data and processes the data inorder to determine sound parameters. Thus, the machine or transformationtest is fulfilled and that the idea is not abstract.

At least parts of the methods according to the invention may be computerimplemented. Accordingly, the present invention may take the form of anentirely hardware embodiment, an entirely software embodiment (includingfirmware, resident software, micro-code, etc.) or an embodimentcombining software and hardware aspects that may all generally bereferred to herein as a “circuit”, “module” or “system”.

Furthermore, the present invention may take the form of a computerprogram product embodied in any tangible medium of expression havingcomputer usable program code embodied in the medium.

It can be easily recognized, by one skilled in the art, that theaforementioned method for stringed instruments' pickup may be performedand/or controlled by one or more computer programs. Such computerprograms are typically executed by utilizing the computing resources ina computing device. Applications are stored on a non-transitory medium.An example of a non-transitory medium is a non-volatile memory, forexample a flash memory while an example of a volatile memory is RAM. Thecomputer instructions are executed by a processor. These memories areexemplary recording media for storing computer programs comprisingcomputer-executable instructions performing all the steps of thecomputer-implemented method according the technical concept presentedherein.

While the invention presented herein has been depicted, described, andhas been defined with reference to particular preferred embodiments,such references and examples of implementation in the foregoingspecification do not imply any limitation on the invention. It will,however, be evident that various modifications and changes may be madethereto without departing from the broader scope of the technicalconcept. The presented preferred embodiments are exemplary only, and arenot exhaustive of the scope of the technical concept presented herein.

Accordingly, the scope of protection is not limited to the preferredembodiments described in the specification, but is only limited by theclaims that follow.

uts of at least half-period of a given periodic function.

1. A method for stringed instruments' pickup the method comprising astep of: capturing images of mechanical vibrations of at least onestring; and converting them to a signal representative of a string'scurrent state the method being characterized in that the capturingcomprises the following steps: capturing, using an image capturingdevice, image frames comprising views of at least one musicalinstrument's string in a still state; storing the captured image as areference of a still state; capturing (500), using the image capturingdevice, image frames comprising views of at least one musicalinstrument's string in a vibrating state; storing the captured image asa reference of a vibrating state; comparing (520) the still statereference with a vibrating state reference in order to find (530)amplitude of vibrations of each string as well as frequency of eachstring vibrations based on amplitude height in pixels with reference tothe still state and determining a frequency of each vibrating string onthe basis of the number of pixels between two nuts of at leasthalf-period of a given periodic function.
 2. The method according toclaim 1 wherein the rate with which the frames are delivered iscontrolled by a clock (450) and is at least twice as high as the highestfrequency a given musical stringed instrument is able to produce.
 3. Themethod according to claim 1 wherein before the comparing step (520), animage processing step (510) is executed where the irrelevant elements ofthe captured scene as well as the elements which carry meaningfulinformation are identified.
 4. The method according to claim 1 whereinthe method further comprises a step (540) wherein the obtainedinformation of amplitude and frequency of at least one vibrating stringis matched with corresponding MIDI messages that are capable of drivingexternal MIDI sound modules or sound synthesis modules.
 5. The methodaccording to claim 1 wherein the given periodic function is a sine orcosine.
 6. The method according to claim 1 wherein the image capturingdevice viewing axis (202C) creates an angle (203C, 203B) with the stillstring axis (201, 607, 707) in the range of 0 to 90 degrees.
 7. Themethod according to claim 1 wherein determining the frequency includescalculating a time per pixel on the basis of a known calibratingfrequency of a vibrating string, and a number of pixels between twonodes (613) of half-period and applying the following formula:$f_{Sound} = \frac{1}{2 \cdot ( {t_{{Node}\mspace{14mu} 2} - t_{{Node}\mspace{14mu} 1}} )}$8. The method according to claim 1 wherein the comparing step is basedon a correlation of a still string axis (201, 607, 707), and the cameraviewing axis (202A, 202B, 202C), and the most distant pixel's (616A,616B, 616C, 616D) trajectory axis.
 9. The method according to claim 1wherein the information about the sound frequency and amplitude,produced by the stringed instrument, takes place in time intervals equalto, at least, the full vibration period of the identified tone.
 10. Anon-transitory computer readable medium storing computer-executableinstructions performing all the steps of the computer-implemented methodaccording to claim 1 when executed on a computer.
 11. A system forstringed instruments' pickup the system comprising: a video cameramodule (111); a video data analysis module (112) configured to executedall step of the method according to claim 1; and output module (113)outputting frequency and amplitude of at least one vibrating string inthe musical instrument.